Standardize parsing of symbol and string array literals#16748
Draft
straight-shoota wants to merge 6 commits intocrystal-lang:masterfrom
Draft
Standardize parsing of symbol and string array literals#16748straight-shoota wants to merge 6 commits intocrystal-lang:masterfrom
straight-shoota wants to merge 6 commits intocrystal-lang:masterfrom
Conversation
ysbaddaden
approved these changes
Mar 19, 2026
Co-authored-by: Julien Portalier <julien@portalier.com>
straight-shoota
added
the
breaking-change
Member
Author
|
An ecosystem test showed that {% begin %}
%q(%t{)
{% end %}This seems to be a regression from #16672 (without it, this patch should work fine). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, string array literal parsing used a custom implementation, combined with a custom lexer mode (
next_string_array_token).This patch changes that to reuse the same parser and lexer features as other string literals. This standardization removes an extra concept and thus reduces code surface area. In return, we're adding some conditionals for this specific literal type into the standard implementations. This increases complexity a little bit, but it's still fairly easy to reason about and a common pattern in the parser.
Combining the parsing of all string-related literals clearly shows their differences (in the form of such conditionals) and ensures consistent behaviour.
In particular, this change establishes standard behaviour for escape characters, and thus fixes #12277
However, it's unclear whether we accept this behaviour change (see #12277 (comment)). If not, we need to implement a non-standard escape algorithm to ensure backwards-compatibility.
The main motivation for this refactor is that it opens the path for implementing string literals with interpolation (RFC 0021).
Resolves #12277